The present invention is related to a foveated display. More specifically, the present invention is related to a foveated display where a detailed imaging element, a focusing mechanism and a wide area imaging element have no moving parts.
Digital displays generally have much coarser resolution than the human eye is capable of resolving. The human eye has a central xe2x80x9cfovealxe2x80x9d region in which up to 60 linear dots can be resolved in one angular degree (ie: the human foveal region has an acuity of 60 dots per linear degree). This is surrounded by a region known as the macula, which has somewhat less acuity, that extends to about four angular degrees. Surrounding the macula, the retina consists of a wide region having relatively low acuity, extending approximately 170 angular degrees for each eye (C., Curcio, K. Sloan, O. Packer, A. Hendrickson, R. Kalina, (1987) xe2x80x9cDistribution of cones in human and monkey retina: individual variability and radial asymmetry.xe2x80x9d Science 236, 579-582), incorporated by reference herein.
The human eyeball is capable of rotating rapidly about its center, and thereby rapidly repositioning its fovea, at an angular velocity of approximately 700 degrees per second. Such rapid movements are called saccades. The brain integrates over time the information entering the small but very agile high resolution-density foveal region, to create the subjective impression of a uniform-density very wide angle and high resolution display. To the human observer, the subjective impression is that our eyes can perceive 60 linear dots for every angular degree, or about 60xc3x97170=102,000 linear dots across the eye""s entire field of vision. Over the area visible to the eye from a fixed head position, this is equivalent to a digital display having on the order of 100,000,000,000 pixels.
In order for a static display to fully exploit the acuity of human vision, it would be necessary to combine tens of thousands of conventional displays into a giant mosaic. This would be very expensive and unwieldy.
A number of researchers have created displays for which resolution varies over the display surface, to better take advantage of the human eye""s foveal architecture. Such foveated displays contain a higher resolution in their center than near their periphery. If the user looks directly at the center of such a display, he can receive more information per display pixel than is the case for uniform density displays. The earliest foveal displays required the eye gaze to be fixed in a single direction in which the resolution was highest, for an extended period of time. This is useful for experimental measurements of human visual acuity, but is not practical for most uses, as humans find it very uncomfortable to maintain a fixed gaze direction for a long period of time.
Levoy et al., incorporated by reference herein, has tracked a user""s head position, using a rotating mirror to actively reposition a small high resolution-density display, within a fixed larger display, to accommodate the high acuity in the foveal region. The larger display was used to cover a large part of the visual field at relatively low acuity. This device followed the user as he rotated his head, but it required the user to gaze straight ahead, since the high density region corresponded to the direction the user""s head was facing, not to his gaze direction.
A number of researchers have developed effective technologies to continually track human gaze (S. Baluja, D. Pomerleau, xe2x80x9cNon-intrusive gaze-tracking using artificial neural networks.xe2x80x9d Neural information processing systems 6, Morgan Kaufman Publishers, New York, 1994), (J. Hansen, A Andersen, P. Roed, xe2x80x9cEye-gaze control of multimedia systems.xe2x80x9d In Y. Anzai, K. Ogawa and H. Mori (eds), Symbiosis of human and artifact. Proceedings of the 6th international conference on human computer interaction. Elsevier Science Publisher, Amsterdam, 1995), (R. Jacob, xe2x80x9cEye tracking in advanced interface design.xe2x80x9d In W. Barfield and T. Furness (eds.), Advanced interface design and virtual environments. Oxford University Press, Oxford, 1995), (R. Stiefelhagen, J. Yang, A. Waibel, xe2x80x9cTracking Eyes and Monitoring Eye Gaze,xe2x80x9d Workshop on Perceptual User Interfaces, Banff, Canada, 1997), all of which are incorporated by reference herein. Tracked gaze direction has long been used to vary spatially across a displayed image between low and moderate resolution (U.S. Pat. No. 4,348,186xe2x80x94Pilot helmet mounted CIG display with eye coupled area of interest), incorporated by reference herein, where moderate resolution is defined to be the highest resolution that can be effected in a single raster display device (e.g.: 1600xc3x971200 pixels). At normal viewing distances, such resolutions perform very poorly in comparison with the acuity of which the human visual system is capable.
Gaze direction information from a tracking device could be used to present a person with a very high resolution image in the foveal region, by constructing a display apparatus which continually repositions a high resolution-density image, depending on the direction of the observer""s gaze. This is a much more difficult engineering task to achieve with speed, accuracy and low cost than is head-orientation tracking, as gaze saccades are considerably more rapid than are changes in head direction.
One method of positioning a high resolution-density image so as to match a rapidly changing foveal region is to mechanically rotate a set of mirrors lying in a single optical path that contains a large surrounding lower resolution image which has been optically combined with a small higher resolution image. The combined image is projected through a common optical system, into the direction in which a gaze-tracked observer is gazing (U.S. Pat. No. 4,634,384: Head and/or eye tracked optically blended display system), incorporated by reference herein.
Another related method, more suitable for a head-mounted display, is to use a half-silvered mirror and retroreflective material, so as to position a rotating mirror to be coincident with the center of the observer""s eyeball in the optical path (B. Bederson, R. Wallace, E. Schwartz, xe2x80x9cA miniature pan-tilt actuator: the spherical pointing motor,xe2x80x9d IEEE Transactions Robotics and Automation, vol. 10, pp. 298-308, 1994), incorporated by reference herein. A foveated display is positioned in front of this mirror. Changes in the direction of the observer""s gaze by angle xcex8 about any axis are mimicked by rotating the mirror by angle xcex8/2, thereby attempting to maintain optical alignment between the observer""s gaze and the high-resolution center of the foveated display.
The major shortcoming of this approach is the need for a mechanically moving mirror, which, being mechanical, is subject to all of the attendant problems of accuracy, vibration, calibration, drift, and unwanted resonance.
Similarly, foveated camera sensors have been described by (P. Kortum, W. Geisler, xe2x80x9cImplementation of a foveated image-coding system for bandwidth reduction of video images,xe2x80x9d SPIE Proceedings: Human Vision and Electronic Imaging, vol. 2657, pp. 350-360, 1996), (F. Pardo, J. A. Boluda, J. J Perez, B. Dierickx, D. Scheffer, xe2x80x9cDesign issues on CMOS space-variant image sensors,xe2x80x9d Proc. SPIE, Advanced Focal Plane Processing and Electronic Cameras, Vol. 2950,pp. 98-107, 1996), (J. van der Spiegel, G. Kreider, C. Claeys, I. Debusschere, G. Sandini, P. Dario, F. Fantini, P. Belluti, G. Soncini, xe2x80x9cA foveated retina-like sensor using CCD technology,xe2x80x9d. In C. Mead and M. Ismail, editor, Analog VLSI implementation of neural systems, chapter 8, pp. 189-212. Kluwer Academic Publishers, Boston, 1989. Proceedings of a workshop on Analog Integrated Neural Systems), (R. Wodnicki, G. W. Roberts and M. D. Levine, xe2x80x9cA foveated image sensor in standard CMOS technology,xe2x80x9d Proc. Custom Integrated Circuits Conf., pp. 357-360, 1995), all of which are incorporated by reference herein, and others. These sensors have radially varying spatial acuity, generally attempting to emulate the human visual system""s logarithmic drop-off of acuity with radial distance from its foveal region. In the current art, such sensors can vary their location of highest acuity only by moving either a set of mirrors, or else by mechanically rotating the sensor chip itself, along with its associated optics.
As is the case with foveated image projection, this approach is subject to all of the attendant problems of accuracy, vibration, calibration, drift, and unwanted resonance that result from the requirement for rapidly moving mechanical components.
The present invention pertains to a foveated display. The display comprises a computer. The display comprises a detailed imaging element for imaging a detailed portion of an image from the computer. The imaging element is connected to the computer. The display comprises a focusing mechanism for focusing the detailed portion of the image. The focusing mechanism is disposed adjacent to the imaging element and connected to the computer. The display comprises a wide area imaging element for imaging a wide-area portion of the image. The wide area imaging element is connected to the computer. The display comprises means to optically combine the detailed portion with the wide area portion to form a foveated image. The display comprises an imaging lens which focuses the foveated image to a desired location wherein the detailed imaging element, the focusing mechanism and the wide area imaging element have no moving parts.
The present invention pertains to a method for producing a foveated image. The method comprises the steps of producing a detailed portion of the image with a detailed imaging element from a computer connected to the detailed imaging element. Then there is the step of producing a wide area portion of the image with a wide area imaging element from the computer connected to the wide area imaging element. Next there is the step of combining the detailed portion with the wide area portion to form the foveated image. Then there is the step of focusing the foveated image onto a desired location.